Digital motion picture restoration involves a variety of image processing operations that require a considerable amount of computations and take a relatively long amount of time, even with powerful computers. An example of a computation-intensive image processing technique is motion estimation, which is required for high fidelity repair. Generally, motion estimation allows a defective region of an image to be replaced with a corresponding image region from a previous or subsequent image frame.
Longer processing times are acceptable if the computations are conducted automatically, without user interaction. Numerous complex algorithms have been developed for automatic image sequence restoration. For a detailed discussion of such image restoration techniques, see, for example, Anil Kokaram, “Motion Picture Restoration,” (Springer, 1998). Currently available image restoration techniques, however, do not provide a complete solution. Generally, currently available image restoration techniques suffer from incomplete repair or false detection (or both), resulting in image artifacts. Thus, human intervention is eventually required to complete the image restoration and obtain high quality artifact-free restoration.
FIG. 1 illustrates a conventional one-step image restoration process 100, where an operator 110 works with an unprocessed, original image sequence 120 to produce a repaired image sequence 140. The image restoration process shown in FIG. 1 has been proposed, for example, for use in the image restoration software and services offered by Mathematical Technologies, Inc. of Providence, R.I. Generally, the operations performed during an interactive session 130 are conducted under the supervision of the operator 110 who manually identifies defects in the original image sequence 120. Thus, the image restoration process 100 does not make efficient use of expensive operator time. Once defects are identified by the operator in this manner, well-known restoration algorithms are typically applied during the interactive session 130 to generate the repaired image sequence 140.
FIG. 2 illustrates a partially automated image restoration process 200. As shown in FIG. 2, the operator 210 initially specifies parameters 215 that control the automatic detection and repair of defects in the original image sequence 220 during an automatic repair stage 230. The operator 210 can review the repaired image sequence 240 generated by the automatic repair stage 230 together with the original image sequence 220, and accept, reject or modify any of the automatic repairs during an interactive session 250 to produce a repaired image sequence 260. An example of the partially automated image restoration process 200 shown in FIG. 2 is the Revival™ software package commercially available from da Vinci Systems, Inc. of Fort Lauderdale, Fla.
While the partially automated image restoration process 200 automatically detects and repairs images without human interaction in a more efficient manner than the one-step image restoration process 100 of FIG. 1, the operator 210 still has very limited abilities to mark and repair additional defects that are not automatically detected during the automatic repair 230. Thus, it is difficult for the operator 210 to achieve a high quality image restoration during the interactive session 250 because the intermediate results from the automatic repair processing 230, such as motion estimation and granularity information, are not available. A need therefore exists for a method and apparatus for removing defects from images that improve the quality of the image repair while also making efficient use of the operator's time.